Pandas Profiling Report for Nhabe Radar dataset

Dataset statistics

Number of variables	8
Number of observations	1414000
Missing cells	4831148
Missing cells (%)	42.7%
Duplicate rows	138
Duplicate rows (%)	< 0.1%
Total size in memory	86.3 MiB
Average record size in memory	64.0 B

Variable types

Numeric	8

Alerts

Dataset has 138 (< 0.1%) duplicate rows	Duplicates
`altitude` is highly overall correlated with `time`	High correlation
`latitude` is highly overall correlated with `velocity`	High correlation
`reflectivity` is highly overall correlated with `total_power`	High correlation
`spectrum_width` is highly overall correlated with `time`	High correlation
`time` is highly overall correlated with `altitude` and 1 other fields	High correlation
`total_power` is highly overall correlated with `reflectivity`	High correlation
`velocity` is highly overall correlated with `latitude`	High correlation
`reflectivity` has 1220984 (86.3%) missing values	Missing
`total_power` has 1046362 (74.0%) missing values	Missing
`velocity` has 1264638 (89.4%) missing values	Missing
`spectrum_width` has 1299164 (91.9%) missing values	Missing

Reproduction

Analysis started	2024-04-22 10:23:27.187092
Analysis finished	2024-04-22 10:23:38.215237
Duration	11.03 seconds
Software version	ydata-profiling vv4.7.0
Download configuration	config.json

longitude
Real number (ℝ)

Distinct	928070
Distinct (%)	65.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	10.667174

Minimum	9.582827
Maximum	11.736427
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	10.8 MiB

Quantile statistics

Minimum	9.582827
5-th percentile	9.9020188
Q1	10.395782
median	10.663281
Q3	10.943005
95-th percentile	11.425909
Maximum	11.736427
Range	2.1536
Interquartile range (IQR)	0.54722262

Descriptive statistics

Standard deviation	0.43556567
Coefficient of variation (CV)	0.040832341
Kurtosis	-0.27982083
Mean	10.667174
Median Absolute Deviation (MAD)	0.273649
Skewness	-0.015761651
Sum	15083383
Variance	0.18971745
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
10.65961	2845	0.2%
10.659617	74	< 0.1%
10.659609	61	< 0.1%
10.659618	58	< 0.1%
10.659625	45	< 0.1%
10.659613	45	< 0.1%
10.659626	44	< 0.1%
10.659612	40	< 0.1%
10.659614	39	< 0.1%
10.659611	34	< 0.1%
Other values (928060)	1410715	99.8%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
9.582827	1	< 0.1%
9.582888	1	< 0.1%
9.583026	1	< 0.1%
9.583459	1	< 0.1%
9.583496	1	< 0.1%
9.58354	1	< 0.1%
9.583594	1	< 0.1%
9.583719	1	< 0.1%
9.584062	1	< 0.1%
9.584208	1	< 0.1%

Value	Count	Frequency (%)
11.736427	1	< 0.1%
11.736286	1	< 0.1%
11.736244	1	< 0.1%
11.735795	1	< 0.1%
11.735745	1	< 0.1%
11.735736	1	< 0.1%
11.73558	1	< 0.1%
11.735578	1	< 0.1%
11.735091	1	< 0.1%
11.735078	1	< 0.1%

latitude
Real number (ℝ)

HIGH CORRELATION

Distinct	263081
Distinct (%)	18.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	106.73372

Minimum	105.63263
Maximum	107.82406
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	10.8 MiB

Quantile statistics

Minimum	105.63263
5-th percentile	105.9504
Q1	106.45373
median	106.73201
Q3	107.01683
95-th percentile	107.50838
Maximum	107.82406
Range	2.19143
Interquartile range (IQR)	0.5631

Descriptive statistics

Standard deviation	0.44577492
Coefficient of variation (CV)	0.0041765143
Kurtosis	-0.29988924
Mean	106.73372
Median Absolute Deviation (MAD)	0.28155
Skewness	-0.020378742
Sum	1.5092148 × 10⁸
Variance	0.19871528
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
106.728325	2865	0.2%
106.728294	156	< 0.1%
106.730515	92	< 0.1%
106.72855	92	< 0.1%
106.72829	82	< 0.1%
106.730446	77	< 0.1%
106.7283	75	< 0.1%
106.72832	75	< 0.1%
106.72828	75	< 0.1%
106.728096	75	< 0.1%
Other values (263071)	1410336	99.7%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
105.63263	1	< 0.1%
105.63274	1	< 0.1%
105.63286	1	< 0.1%
105.63318	1	< 0.1%
105.63327	1	< 0.1%
105.633385	1	< 0.1%
105.63342	1	< 0.1%
105.633484	1	< 0.1%
105.63381	1	< 0.1%
105.63396	1	< 0.1%

Value	Count	Frequency (%)
107.82406	1	< 0.1%
107.82397	1	< 0.1%
107.823814	1	< 0.1%
107.82356	1	< 0.1%
107.82337	1	< 0.1%
107.823265	1	< 0.1%
107.82324	1	< 0.1%
107.82312	1	< 0.1%
107.82285	1	< 0.1%
107.82279	1	< 0.1%

altitude
Real number (ℝ)

HIGH CORRELATION

Distinct	23270
Distinct (%)	1.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	5616.9502

Minimum	10
Maximum	25697
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	10.8 MiB

Quantile statistics

Minimum	10
5-th percentile	242
Q1	1434
median	3774
Q3	8082
95-th percentile	17578
Maximum	25697
Range	25687
Interquartile range (IQR)	6648

Descriptive statistics

Standard deviation	5472.9762
Coefficient of variation (CV)	0.97436794
Kurtosis	1.4197677
Mean	5616.9502
Median Absolute Deviation (MAD)	2786
Skewness	1.373846
Sum	7.9423676 × 10⁹
Variance	29953469
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
10	2828	0.2%
158	832	0.1%
30	772	0.1%
121	763	0.1%
92	720	0.1%
604	676	< 0.1%
59	634	< 0.1%
306	600	< 0.1%
40	596	< 0.1%
29	569	< 0.1%
Other values (23260)	1405010	99.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
10	2828	0.2%
11	101	< 0.1%
12	189	< 0.1%
13	253	< 0.1%
14	153	< 0.1%
15	178	< 0.1%
16	495	< 0.1%
17	160	< 0.1%
18	180	< 0.1%
19	413	< 0.1%

Value	Count	Frequency (%)
25697	1	< 0.1%
25686	1	< 0.1%
25658	1	< 0.1%
25644	1	< 0.1%
25636	2	< 0.1%
25633	1	< 0.1%
25619	2	< 0.1%
25613	1	< 0.1%
25605	1	< 0.1%
25602	1	< 0.1%

reflectivity
Real number (ℝ)

HIGH CORRELATION MISSING

Distinct	5700
Distinct (%)	3.0%
Missing	1220984
Missing (%)	86.3%
Infinite	0
Infinite (%)	0.0%
Mean	6.7126096

Minimum	-23.69
Maximum	57.07
Zeros	44
Zeros (%)	< 0.1%
Negative	56221
Negative (%)	4.0%
Memory size	10.8 MiB

Quantile statistics

Minimum	-23.69
5-th percentile	-13.43
Q1	-1.7
median	6.9
Q3	15.2625
95-th percentile	26.18
Maximum	57.07
Range	80.76
Interquartile range (IQR)	16.9625

Descriptive statistics

Standard deviation	11.862737
Coefficient of variation (CV)	1.7672317
Kurtosis	-0.67080317
Mean	6.7126096
Median Absolute Deviation (MAD)	8.48
Skewness	-0.022074791
Sum	1295641
Variance	140.72452
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
8.88	90	< 0.1%
6.76	88	< 0.1%
10.02	88	< 0.1%
6.63	87	< 0.1%
8.28	86	< 0.1%
9.53	85	< 0.1%
7.23	84	< 0.1%
10.38	83	< 0.1%
6.84	82	< 0.1%
7.62	82	< 0.1%
Other values (5690)	192161	13.6%
(Missing)	1220984	86.3%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-23.69	1	< 0.1%
-23.02	1	< 0.1%
-22.84	1	< 0.1%
-22.67	1	< 0.1%
-22.41	1	< 0.1%
-22.31	1	< 0.1%
-22.08	1	< 0.1%
-22.06	1	< 0.1%
-21.91	2	< 0.1%
-21.86	1	< 0.1%

Value	Count	Frequency (%)
57.07	1	< 0.1%
53.27	1	< 0.1%
52.57	1	< 0.1%
52.56	1	< 0.1%
52.15	1	< 0.1%
50.35	1	< 0.1%
43.16	1	< 0.1%
42.85	1	< 0.1%
42	1	< 0.1%
40.77	2	< 0.1%

total_power
Real number (ℝ)

HIGH CORRELATION MISSING

Distinct	7769
Distinct (%)	2.1%
Missing	1046362
Missing (%)	74.0%
Infinite	0
Infinite (%)	0.0%
Mean	10.400532

Minimum	-19.74
Maximum	78.98
Zeros	97
Zeros (%)	< 0.1%
Negative	77157
Negative (%)	5.5%
Memory size	10.8 MiB

Quantile statistics

Minimum	-19.74
5-th percentile	-7.31
Q1	1.44
median	9.24
Q3	18.1
95-th percentile	31.14
Maximum	78.98
Range	98.72
Interquartile range (IQR)	16.66

Descriptive statistics

Standard deviation	12.160026
Coefficient of variation (CV)	1.1691735
Kurtosis	0.6941188
Mean	10.400532
Median Absolute Deviation (MAD)	8.27
Skewness	0.636048
Sum	3823630.7
Variance	147.86624
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
9.11	161	< 0.1%
6.39	156	< 0.1%
5.03	149	< 0.1%
7.13	149	< 0.1%
7.45	149	< 0.1%
7.56	148	< 0.1%
9.62	147	< 0.1%
7	146	< 0.1%
9.85	145	< 0.1%
7.71	145	< 0.1%
Other values (7759)	366143	25.9%
(Missing)	1046362	74.0%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-19.74	1	< 0.1%
-18.52	1	< 0.1%
-18.48	1	< 0.1%
-18.27	1	< 0.1%
-18.02	1	< 0.1%
-17.81	1	< 0.1%
-17.69	1	< 0.1%
-17.65	1	< 0.1%
-17.64	1	< 0.1%
-17.57	1	< 0.1%

Value	Count	Frequency (%)
78.98	1	< 0.1%
78.46	1	< 0.1%
77.1	1	< 0.1%
76.72	1	< 0.1%
76.28	1	< 0.1%
76.05	1	< 0.1%
75.77	1	< 0.1%
75.73	1	< 0.1%
75.68	1	< 0.1%
75.06	1	< 0.1%

velocity
Real number (ℝ)

HIGH CORRELATION MISSING

Distinct	4395
Distinct (%)	2.9%
Missing	1264638
Missing (%)	89.4%
Infinite	0
Infinite (%)	0.0%
Mean	-1.865443

Minimum	-31.6
Maximum	31.6
Zeros	52
Zeros (%)	< 0.1%
Negative	93600
Negative (%)	6.6%
Memory size	10.8 MiB

Quantile statistics

Minimum	-31.6
5-th percentile	-11.8
Q1	-7.48
median	-2.74
Q3	3.31
95-th percentile	10.53
Maximum	31.6
Range	63.2
Interquartile range (IQR)	10.79

Descriptive statistics

Standard deviation	7.4794865
Coefficient of variation (CV)	-4.0094961
Kurtosis	1.0127801
Mean	-1.865443
Median Absolute Deviation (MAD)	5.17
Skewness	0.4062626
Sum	-278626.3
Variance	55.942718
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-7.27	138	< 0.1%
-7.33	133	< 0.1%
-7.08	130	< 0.1%
-7.55	127	< 0.1%
-7.39	127	< 0.1%
-6.33	124	< 0.1%
-6.66	123	< 0.1%
-7.1	122	< 0.1%
-7.26	120	< 0.1%
-7.24	119	< 0.1%
Other values (4385)	148099	10.5%
(Missing)	1264638	89.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-31.6	1	< 0.1%
-31.59	2	< 0.1%
-31.58	3	< 0.1%
-31.57	2	< 0.1%
-31.56	2	< 0.1%
-31.51	1	< 0.1%
-31.5	1	< 0.1%
-31.49	1	< 0.1%
-31.48	2	< 0.1%
-31.46	4	< 0.1%

Value	Count	Frequency (%)
31.6	1	< 0.1%
31.59	1	< 0.1%
31.57	1	< 0.1%
31.56	1	< 0.1%
31.54	6	< 0.1%
31.53	1	< 0.1%
31.52	1	< 0.1%
31.51	3	< 0.1%
31.5	1	< 0.1%
31.49	1	< 0.1%

spectrum_width
Real number (ℝ)

HIGH CORRELATION MISSING

Distinct	662
Distinct (%)	0.6%
Missing	1299164
Missing (%)	91.9%
Infinite	0
Infinite (%)	0.0%
Mean	1.0823093

Minimum	0.01
Maximum	8.84
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	10.8 MiB

Quantile statistics

Minimum	0.01
5-th percentile	0.24
Q1	0.57
median	0.85
Q3	1.32
95-th percentile	2.81
Maximum	8.84
Range	8.83
Interquartile range (IQR)	0.75

Descriptive statistics

Standard deviation	0.83274804
Coefficient of variation (CV)	0.7694178
Kurtosis	5.9382689
Mean	1.0823093
Median Absolute Deviation (MAD)	0.34
Skewness	2.0638622
Sum	124288.07
Variance	0.69346929
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0.01	3086	0.2%
0.58	1093	0.1%
0.66	1092	0.1%
0.69	1092	0.1%
0.68	1086	0.1%
0.65	1085	0.1%
0.64	1074	0.1%
0.61	1068	0.1%
0.73	1065	0.1%
0.54	1062	0.1%
Other values (652)	102033	7.2%
(Missing)	1299164	91.9%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0.01	3086	0.2%
0.02	15	< 0.1%
0.03	15	< 0.1%
0.04	27	< 0.1%
0.05	32	< 0.1%
0.06	48	< 0.1%
0.07	53	< 0.1%
0.08	60	< 0.1%
0.09	74	< 0.1%
0.1	82	< 0.1%

Value	Count	Frequency (%)
8.84	1	< 0.1%
8.63	1	< 0.1%
8.47	1	< 0.1%
8.25	1	< 0.1%
7.62	1	< 0.1%
7.5	1	< 0.1%
7.41	2	< 0.1%
7.32	1	< 0.1%
7.29	1	< 0.1%
7.24	1	< 0.1%

time
Real number (ℝ)

HIGH CORRELATION

Distinct	316
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	175.98656

Minimum	1
Maximum	347
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	10.8 MiB

Quantile statistics

Minimum	1
5-th percentile	22
Q1	92
median	177
Q3	264
95-th percentile	332
Maximum	347
Range	346
Interquartile range (IQR)	172

Descriptive statistics

Standard deviation	99.438048
Coefficient of variation (CV)	0.56503205
Kurtosis	-1.1854954
Mean	175.98656
Median Absolute Deviation (MAD)	86
Skewness	0.0047503565
Sum	2.48845 × 10⁸
Variance	9887.9255
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
292	5500	0.4%
331	5500	0.4%
310	5500	0.4%
112	5500	0.4%
223	5500	0.4%
231	5500	0.4%
141	5500	0.4%
104	5500	0.4%
283	5500	0.4%
39	5500	0.4%
Other values (306)	1359000	96.1%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
1	4500	0.3%
2	5000	0.4%
3	4000	0.3%
4	4000	0.3%
5	4500	0.3%
6	5000	0.4%
7	500	< 0.1%
12	1000	0.1%
13	5000	0.4%
14	5000	0.4%

Value	Count	Frequency (%)
347	4000	0.3%
346	4500	0.3%
345	4500	0.3%
344	4500	0.3%
343	5000	0.4%
342	4500	0.3%
341	4500	0.3%
340	3500	0.2%
339	5000	0.4%
338	5000	0.4%

Auto

Heatmap
Table

	altitude	latitude	longitude	reflectivity	spectrum_width	time	total_power	velocity
altitude	1.000	-0.011	-0.018	0.342	-0.427	0.679	-0.233	-0.051
latitude	-0.011	1.000	-0.018	-0.145	-0.060	-0.016	-0.047	0.854
longitude	-0.018	-0.018	1.000	0.387	-0.191	-0.054	0.257	0.050
reflectivity	0.342	-0.145	0.387	1.000	-0.134	-0.089	0.801	-0.101
spectrum_width	-0.427	-0.060	-0.191	-0.134	1.000	-0.514	0.036	0.035
time	0.679	-0.016	-0.054	-0.089	-0.514	1.000	-0.181	0.147
total_power	-0.233	-0.047	0.257	0.801	0.036	-0.181	1.000	-0.034
velocity	-0.051	0.854	0.050	-0.101	0.035	0.147	-0.034	1.000

Count
Matrix

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

First rows
Last rows

	longitude	latitude	altitude	reflectivity	total_power	velocity	spectrum_width	time
0	10.659610	106.728325	10.0	NaN	17.01	NaN	NaN	25.0
1	10.661768	106.728340	11.0	10.37	22.41	NaN	NaN	25.0
2	10.663927	106.728355	13.0	22.60	46.00	NaN	NaN	25.0
3	10.666085	106.728360	14.0	NaN	50.21	NaN	NaN	25.0
4	10.668242	106.728370	16.0	NaN	49.70	NaN	NaN	25.0
5	10.670402	106.728386	17.0	12.99	35.05	0.06	1.41	25.0
6	10.672560	106.728400	19.0	13.82	36.21	-4.62	0.01	25.0
7	10.674717	106.728410	21.0	24.64	30.71	26.28	1.19	25.0
8	10.676876	106.728424	22.0	12.22	25.30	25.56	1.48	25.0
9	10.679034	106.728430	24.0	17.96	32.42	-4.91	0.34	25.0

	longitude	latitude	altitude	reflectivity	total_power	velocity	spectrum_width	time
1413990	11.691405	106.710144	24994.0	NaN	NaN	NaN	NaN	342.0
1413991	11.693505	106.710106	25047.0	NaN	NaN	NaN	NaN	342.0
1413992	11.695603	106.710070	25099.0	NaN	NaN	NaN	NaN	342.0
1413993	11.697704	106.710030	25152.0	NaN	NaN	NaN	NaN	342.0
1413994	11.699803	106.709990	25205.0	NaN	NaN	NaN	NaN	342.0
1413995	11.701903	106.709950	25257.0	NaN	NaN	NaN	NaN	342.0
1413996	11.704002	106.709915	25310.0	NaN	NaN	NaN	NaN	342.0
1413997	11.706101	106.709885	25362.0	NaN	NaN	NaN	NaN	342.0
1413998	11.708201	106.709850	25415.0	NaN	NaN	NaN	NaN	342.0
1413999	11.710299	106.709810	25468.0	NaN	NaN	NaN	NaN	342.0

Most frequently occurring

	longitude	latitude	altitude	reflectivity	total_power	velocity	spectrum_width	time	# duplicates
104	10.65961	106.728325	10.0	NaN	16.66	NaN	NaN	269.0	4
42	10.65961	106.728325	10.0	NaN	16.08	NaN	NaN	330.0	3
108	10.65961	106.728325	10.0	NaN	16.70	NaN	NaN	281.0	3
0	10.65961	106.728325	10.0	NaN	15.67	NaN	NaN	90.0	2
1	10.65961	106.728325	10.0	NaN	15.68	NaN	NaN	54.0	2
2	10.65961	106.728325	10.0	NaN	15.71	NaN	NaN	54.0	2
3	10.65961	106.728325	10.0	NaN	15.75	NaN	NaN	95.0	2
4	10.65961	106.728325	10.0	NaN	15.76	NaN	NaN	175.0	2
5	10.65961	106.728325	10.0	NaN	15.80	NaN	NaN	153.0	2
6	10.65961	106.728325	10.0	NaN	15.81	NaN	NaN	32.0	2

Overview

Variables

Interactions

Correlations

Missing values

Sample

Duplicate rows

Most frequently occurring